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I .  INTRODUCTION 

A.   BACKGROUND 

A  major  problem  for  the  U  S  Navy  flight  program  is  pilot 
workload  saturation  in  the  cockpit.   This  creates  a 
situation  in  which  there  is  no  reserve  to  deal  with 
unforeseen  contingencies  and  no  room  to  add  other  needed 
tasks.   The  Naval  Air  Development  Center  (NADC)  located  at 
Warminster,  Pennsylvania,  is  particularly  interested  in 
finding  a  solution  which  can  be  implemented  without  creating 
a  corresponding  degradation  in  the  performance  of  the 
individual  aircraft's  mission  [Interview  with  Warner,  1987]. 

Research  in  the  area  of  dual-task  performance  indicates 
four  ways  of  tackling  this  problem  [ Wickens, 1981 ] . 

1.  Training  pilots  to  perform  multiple  tasks  more 
efficiently. 

2.  The  use  of  criteria  which  select  those  individuals 
with  more  proficient  time-sharing  skills  in  the 
recruitment  of  pilots. 

3.  Designing  the  cockpits  so  that  the  pilot  will  only  be 
requested  to  concurrently  perform  tasks  which  are  more 
efficiently  shared. 

4.  Design  the  placement  of  controls  and  displays  in  the 
cockpit  to  allow  for  greatest  time-sharing  efficiency. 

The  design  of  the  cockpit  has  been  studied  extensively 

so  that  it  is  doubtful  that  any  further  large  reductions  in 

workload  can  be  achieved  in  that  area.   Pilot  selection 

could  not  be  based  solely  on  time-sharing  abilities  even  if 

techniques  for  accurate  measurement  of  this  ability  were 


currently  available.   Pilot  selection  must  be  based  on  many 
more  important  criteria.   Also,  since  pilots  already  receive 
extensive  training  it  is  doubtful  that  any  significant 
increases  in  efficiency  can  be  achieved  by  providing 
additional  training.   Therefore,  attention  is  being  focused 
on  the  third  alternative  of  trying  to  ensure  that  tasks, 
which  a  pilot  must  perform  concurrently,  are  compatible  for 
the  purpose  of  efficient  time-sharing. 

Recent  studies  indicate  that  the  use  of  voice 
recognition/synthesis  in  an  aircraft  cockpit  may  provide  the 
best  means  of  achieving  the  compatibility  of  tasks  which 
will  achieve  the  desired  goal  of  workload  reduction. 

B.   LITERATURE  SEARCH 

A  large  volume  of  material  dealing  with  operator 
workload  has  been  generated  in  recent  years.   This  interest 
in  operator  workload  is  most  likely  the  result  of  the 
existence  of  jobs  which  require  multiple  tasks  to  be 
performed  in  a  very  complex  environment.   An  excellent 
example  of  such  a  job  is  the  piloting  of  a  U.  S.  Navy 
aircraft . 

Literature  generated  in  the  area  of  human  performance 
covers  all  aspects  of  time-sharing,  display  design,  single 
and  multiple  tasking,  attention  resources,  etc.   Because  of 
this  proliferation  of  research  it  is  necessary  to  limit  the 


review  of  this  material  to  that  of  attention  resources  and 
time-sharing.   These  two  areas  provide  the  theoretical  basis 
for  pursuing  the  use  of  voice  recognition/synthesis  in  an 
aircraft  cockpit  as  a  means  to  reduce  pilot  workload. 

1 .  Attention  as  a  Limited  Resource 

In  1967  Moray  published  a  paper  which  highlighted 
the  quantitative  aspects  of  attention.   He  asserted  that 
attention  should  be  thought  of  as  having  a  limited  capacity 
in  much  the  same  way  that  a  given  computer  is  limited  in  how 
much  data  it  can  process.   Additionally,  he  stated  that  this 
capacity  could  be  divided  into  various  amounts  to  be 
assigned  to  tasks  on  the  basis  of  difficulty  and  demand. 
Particularly  beneficial  was  the  flexibility  and  ability  to 
share  these  resources  that  was  highlighted  in  his  treatment 
of  the  subject.   [ Wickens, 1984 :pp .  292-293] 

2 .  Kahneman  Model 

The  Kahneman  model  is  important  because  it  was  the 
first  attempt  to  predict  performance  based  on  the  limited 
capacity/resource  theory.   However,  his  model  still  dealt 
with  attention  as  having  a  single  source  capacity  and  could 
not  account  for  instances  in  which  more  difficult  secondary 
tasks  interfered  less  with  the  primary  task  than  other  less 
difficult  tasks.   This  seemed  to  indicate  that  the  theory  of 
a  single  resource  was  possibly  too  simplistic. 
[Wickens, 1984  and  Wickens, 1987 ] 


3 .   Multiple-Resource  Theory 

The  single  resource  theory  of  the  Kahneman  Model  was 
valuable  in  asserting  the  idea  of  attention  as  a  limited 
capacity  resource.   However,  it  failed  to  explain 
differences  from  predicted  results  which  occurred  when 
certain  tasks  were  combined.  [Wickens,  1984] 

In  an  attempt  to  resolve  these  apparent 
discrepancies,  Wickens  developed  a  Multiple-Resource  model 
which  combined  not  only  the  theory  of  attention  as  a 
resource  but  also  the  theories  of  attention  as  structural  in 
nature  I Wickens,  1981,  Wickens, undated,  Wickens, 1988  and 
Wickens,  1984 ] . 

Building  on  research  by  individuals  such  as  Pachella 
(1974),  Kinsbourne  and  Hicks  (1978),  Harris,  Owens  and  North 
(1978)  and  others  [Wickens, undated ] ;  Wickens  designed  a 
three  dimensional  model  (Figure  1)  which  describes  attention 
along  the  dimensions  of  (1)  stages  of  processing,  (2) 
cerebral  hemisphere  of  operation  and  (3)  modalities  of 
processing  (encoding  and  response). 

Stages  of  processing  include  the  processes  of 
encoding,  memory  &  transformations,  and  responding.   The 
idea  of  different  sequentially  ordered  stages  of  processing 
has  achieved  a  certain  level  of  acceptance  and  even  though 
it  cannot  be  inferred  that  independent  resources  are  used, 


some  research  in  dual  tasking  seems  to  indicate  it  could  be. 
[Wickens,  undated] 
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Figure  1.   Multiple  Resource  Model 
(Adapted  from  Wickens  (undated)) 
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The  research  done  in  the  area  of  separate  cerebral 
hemispheres  of  operation  argues  strongly  that  the  two 
cerebral  hemispheres  act  as  separate  resource  reservoirs. 
In  particular,  spatial  tasks  along  with  left  hand  controls 
are  handled  by  the  right  hemisphere  of  the  brain  whereas 
verbal  tasks  and  right  hand  controls  are  handled  in  the 
left.   Since  a  slight  additional  amount  of  time  is  needed  in 
order  for  the  brain  to  send  messages  between  the  hemispheres 
of  the  brain,  it  is  considered  advantageous  in  certain 
circumstances  to  keep  processing  of  a  single  task  confined 
to  one  hemisphere.   However,  in  other  situations  it  is 


beneficial  to  use  both  hemispheres  since  competition  for  the 
same  resources  can  cause  a  slower  response  than  that 
resulting  from  hemisphere  crossover  time. 

Some  of  the  clearest  measurements  have  been  along 
the  dimension  of  different  modalities  of  processing  in  which 
auditory  versus  visual  encoding,  and  manual  versus  vocal 
responses  have  been  studied.   Sandry  demonstrated  the  effect 
of  these  modalities  in  her  experiment  conducted  on  the  F-18 
simulator  located  at  the  Naval  Air  Test  Center,  Patuxent 
River,  Maryland  in  1981  [ Sandry, 1982 ] . 
4 .   Sandry  study 

In  her  testing,  Sandry  paired  verbal  and  spatial 
tasks   with  the  continuous  primary  spatial  task  of  flying 
an  aircraft.   By  comparing  reaction  times  of  different 
input/output  modality  pairs,  inferences  concerning  expected 
performance  could  be  made.   In  general,  she  found  that  the 
time  required  to  complete  a  verbal  task  could  be  reduced  by 
changing  to  a  modality  combination  which  utilized  voice 
recognition  and/or  synthesis  while  such  a  combination 
resulted  in  degraded  spatial  task  performance. 


Wickens  defines  verbal  tasks  as  "Tasks  for  which  words, 
language,  or  logical  operations  are  natural  mediators  . . . " . 
Spatial  tasks  are  defined  as  "...  almost  any  task  that  requires 
moving,  positioning,  or  orienting  objects  in  space,  or  performing 
other  analog  transformations,..."  [ Wickens, 1987 : p .  16]. 


Sandry's  study  was  useful  in  two  ways.   First,  she 
directs  attention  immediately  to  those  tasks  which  will  be 
most  benefited  by  voice  implementation.   Second,  because  the 
data  was  gathered  on  the  Navy's  F-18  simulator,  it  can  be 
used  for  an  initial  cut  in  projecting  possible  reaction  time 
savings . 

C.   DEPARTMENT  OF  DEFENSE  STUDIES 

1.  Air  Force  B-52  Bomber  Study  [North  and  Lea, 1982] 
This  study  represents  the  armed  forces*  first 

attempt  to  identify  cockpit  tasks  which  would  be  benefited 
most  by  conversion  from  visual/manual  input/output 
modalities  to  audio/vocal  modalities.   The  approach 
developed  by  Honeywell  dealt  with  cockpit  tasks  on  an 
individual  mission  basis.   A  filtering  system  was  then 
developed  which  screened  and  prioritized  candidate  tasks 
using  a  series  of  objective  and  subjective  criteria. 

2.  Navy  F/A-18  Study  [Mountford,  North,  Metz,  and 

Graf funder, 1982] 

The  Navy  study,  which  was  also  conducted  by 
Honeywell,  had  three  objectives.   They  were  to  (1)  perform  a 
literature  review,  (2)  develop  task  selection  procedures, 
and  (3)  study  ^dialogue'  issues.   Of  primary  interest  to 
this  thesis  is  the  second  objective  of  task  selection.   The 
resulting  procedures  refined  those  developed  as  part  of  the 
B-52  study. 


These  two  studies  performed  by  Honeywell,  although 
valuable,  are  weak  in  two  areas.   First,  determination  of 
technological  feasibility  for  a  particular  candidate  task 
for  voice  implementation  is  done  with  a  weighting  factor. 
It  would  be  much  more  straight  forward  to  use  a  go/no-go 
decision  process  for  this.   When  Warner  at  NADC  attempted  to 
use  the  proposed  filtering  process,  tasks  which  were  not 
technologically  feasible  were  not  successfully  screened  out 
[Interview  With  Warner , 1987 ] . 

Secondly,  the  filtering  process  designed  during 
these  studies  chooses  and  prioritizes  candidate  tasks  on  an 
individual  basis  with  only  a  limited  ability  to  judge  the 
interference  effect  when  multiple  tasks  are  converted  to 
voice.   There  is  no  attempt  to  determine  the  most 
advantageous  combination  of  voice  tasks. 

D.   DEPARTMENT  OF  DEFENSE  FLIGHT  TESTS 

The  military  services,  in  spite  of  some  valid 
reservations,  have  displayed  a  continued  interest  in  voice 
synthesis/recognition  as  a  possible  solution  for  the 
saturated  workload  situation  present  in  the  aircraft 
cockpit.   As  a  result,  flight  tests  have  been  performed,  the 
results  of  which  are  discussed  below. 


1.  F/A-18A  Flight  Tests  [Loikith  and  Hall, 1986] 

The  purpose  of  these  flight  tests  was  to  determine 
the  feasibility  and  pilot  utility  of  voice 

synthesis/recognition.   Results  of  the  tests  indicated  that 
voice  implementation  of  cockpit  tasks  was  feasible  and  could 
be  useful  provided  that  technical  problems  with  the 
particular  voice  system  tested  could  be  corrected.   The  Lear 
Siegler,  Inc;  Voice  Control  Interactive  Device  was  used  for 
these  tests. 

2.  AFTI/F-16  Flight  Tests  [Williamson, undated ] 

Using  the  Texas  Instrument  Voice  Interactive  System 

the  Air  Force  obtained  impressive  results.   The  following 

excerpt  from  the  "Discussion"  portion  of  the  report  reveals 

how  well  this  testing  went. 

The  test  pilots  found  it  safer  and  more  convenient  to  use 
voice  instead  of  traversing  the  awkward  menu  logic  of 
multifunction  keyboards  and  displays.   The  ability  to 
request  information  verbally  and  receive  a  verbal  response 
has  also  received  universal  praise  from  the  pilots. 

This  report  also  highlights  the  fact  that 

development  of  a  method  for  selecting  the  most  promising 

tasks  for  conversion  to  voice  is  necessary. 

3.  JQH-58C  Helicopter  Flight  Tests  [ Szerszynski , 1987 ] 
The  Army  also  experienced  favorable  results  using 

voice  in  the  JOH-58C  helicopter.   One  of  the  sub-tests 


conducted  was  a  simulated  single  pilot  missiorr  during 
which  the  pilot  had  to  simultaneously  control  flight  tasks, 
radio  control,  and  visual  search  .   Their  assessment  of  the 
test  results  was  that  "The  voice  controlled  avionics  system 
is,  based  on  the  data,  the  avionics  control  system  of  choice 
when  the  workload  is  high". 

These  tests  all  demonstrate  the  ability  of  voice 
synthesis/recognition  as  a  means  to  reduce  pilot  workload 
thereby  increasing  efficiency  and  safety.   The  last  two 
tests  in  particular  confirm  the  fact  that  the  necessary 
voice  technology  is  currently  in  existence. 


The  phrase  "simulated   single  pilot  mission"  was  used  since 
a  second  pilot  was  along  for  safety  reasons. 


10 


II .  MODEL 

A.   OVERVIEW 

The  primary  goal  of  this  thesis  is  to  combine  previous 

research  and  DOD  studies  into  a  single  model  which  will 

provide  the  design  engineer  with  guidelines  and  procedures 

for  a  total  systems  approach  to  implementing  voice 

recognition/synthesis  in  the  cockpit.   Wickens  states  the 

goal  of  this  model  best. 

A  major  goal  of  predictive  performance  models  is  to  help 
determine  the  most  effective  design  before  a  system  is 
configured.   Such  models  are  not  likely  to  be  100% 
reliable.   However,  they  may  prevent  the  designer  from 
developing  prototypes  that  will  be  clearly  non-optimal, 
and  will  also  enable  the  designer  to  identify  a  parameter 
space  that  is  most  worthy  of  experimental  investigation. 
[Wickens, 1988] 

Likewise,  this  model  will  provide  the  most  beneficial 
starting  place  for  design  and  testing  but  cannot  be  expected 
to  provide  "the  answer".   Variances  between  aircraft  and 
ever  changing  missions  preclude  any  other  approach. 

This  model  draws  extensively  from  the  methods  developed 
in  the  Honeywell  studies  [North  and  Lea, 1982  and  Mountford, 
North,  Metz  and  Graf f under , 1982 ]  which  use  a  total  systems 
approach  in  deciding  what  systems  to  activate  by  voice.   The 
performance  data  generated  by  Sandry  (1982)  provides  the 
data  for  use  in  projecting  expected  workload  savings.   The 
product  of  this  model  will  allow  the  design  engineer  to 


11 


begin  with  a  first  cut  will  maximize  workload  savings  while 
avoiding  overloading  the  pilot  on  the  voice  interactive 
side . 

To  aid  in  the  future  refinement  of  this  model, 
individual  modules  will  be  used  to  separate  distinct 
portions.   Detailed  descriptions  of  these  modules  follow. 

B.   MODEL  DESCRIPTION 
1 .   Module  One 

a.   Task  Analysis: 

This  portion  of  the  model  was  developed  by 
Honeywell  [North  and  Lea/1982:pp.  8-14  and  Mountford,  North, 
Metz  and  Graf f under , 1982 :pp.  6-18]  as  part  of  two  studies 
performed  for  NADC. 

(1)  Mission  Scenario:   According  to  the 
Honeywell  studies,  the  first  step  in  analyzing  an  aircraft 
cockpit  regarding  the  value  of  implementing  voice 
synthesis/recognition  is  to  develop  a  mission  scenario. 
This  scenario  should  be  a  general  narrative  which  accurately 
describes  the  mission  under  consideration. 

Honeywell  deals  with  mission  scenarios 
individually,  taking  each  mission  separately  and  running  it 
through  the  entire  procedure.   In  this  model  a  broader  look 
is  taken.   All  of  the  major  missions  of  the  particular 
aircraft  under  study  should  be  developed  and  evaluated 
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together  to  provide  an  analysis  which  produces 
recommendations  for  voice  implementation  based  on  the 
aircraft  as  a  total  system. 

In  choosing  the  missions  to  evaluate, 
consideration  should  be  given  to  high  workload  missions 
although  it  is  important  to  include  all  major  missions. 
Infrequently  flown  missions  should  not  be  considered  unless 
unusually  critical  since  they  will  unduly  affect  the 
findings  of  the  evaluation  process. 

(2)  Task  Narrative:   The  second  step  advocated 
by  Honeywell,  is  the  development  of  a  task  narrative  which 
provides  a  detailed  verbal  description  of  each  task  of  the 
mission.   A  task  narrative  should  be  developed  for  each 
mission  scenario.   An  example  of  a  task  narrative  is 
contained  in  the  F-18  Honeywell  study  [North  and  Lea,1982:p 
9]  and  has  been  provided  as  Table  1.   The  tasks  should  be 
listed  in  the  order  in  which  they  occur  during  the  flight. 

(3)  Time-based  Activity  Log;  The  next  step  in 
the  Honeywell  procedure  is  to  use  the  Task  Narrative  to 
"...categorize  each  activity  into  its  human  information 
processing  channels--vocal,  visual,  auditory,  and  manual." 
[Mountford,  North,  Metz,  and  Graf f under , 1982 :p.  10].  Also 
an  index  should  be  included  which  indicates  the  difficulty 
in  distance  and  location  the  pilot  experiences  in  reaching 
for  the  manual  control.   Those  tasks  for  which  the  pilot 
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does  not  reach  would  be  assigned  a  value  of  zero  and 
increase  in  incremental  steps  of  one  up  to  the  most 
difficult.   This  reach  index  is  a  useful  measure  when 
combined  with  other  information  in  deciding  the  best 
candidates  for  speech  synthesis/recognition.   In  Honeywell's 
F-18  study  example  the  highest  value  was  a  three.  Figure  2 
provides  an  example  of  how  this  index  could  be  assigned  to 
an  aircraft  cockpit. 


Table  1.   TASK  NARRATIVE 
(Adapted  From  North  and  Lea  (1982)) 


Pilot  Tasks 


Copilot  Tasks 


Additional  Aircrew  Tasks 


Calls  for  preparation  for 
contact  checklist. 


1.  Establish  radio  contact 


6.  Disengage  autopilot, 


7.  Set  airbrake  lever 
to  position  1 . 


9.  Select  FUR  video. 


Reads  preparation  for 
contact  checklist. 


2.  Check  air  conditioning 
system. 


4.  Turn  ant fcol 1 i sion  lights 
off. 

5.  Turn  naviqat (on 
lights  to  flash. 

6.  Set  slipway  and  airplane 
llnht  switches  to  full 
bright. 

7.  Open  slipway  doors  and 
verify  ready  lights  on. 

8.  Determine  tanker  position 
on  FLIR  sensor. 


10.  Set  air  refueling 
switch  to  ai  r  refuel 


RN — Calls  ranne  to  tanker  in 
1  NM  increments.  Calls 
range  at  3  ...  2  »1. 


N — 3.  Set  rendezvous  radar 
to  standby. 


RN — 8.  Determine  tanker 
position  on  FT.IR 
sensor. 
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ZONE! 


Figure  2.   Anthropometric  Zones 
(Adapted  From  North  and  Lea(1982)) 
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An  example  of  the  Time-based  Activity  Log  is 
included  as  Table  2  [Mountford,  North,  Metz,  and 
Graff under , 1982 :p.  11].   Immediately  obvious  in  this  sample 
is  the  fact  that  many  of  the  tasks  show  input  or  output 
modes  of  "N"  (not  applicable).   This  is  not  appropriate. 
Each  task  can  have  both  input  and  output  modes  specifically 
identified.   For  the  case  where  a  "not  applicable"  entry 
appears  appropriate  the  following  guidelines  should  be 
employed.   If  the  task  is  prompted  by  a  checklist  it  should 
be  considered  to  have  a  visual  input  mode.   If  it  is  a 
memory  item,  then  it  should  be  listed  as  an  audio  input  mode 
because  this  most  closely  identifies  the  information 
processing  channel  (hemisphere)  used.   In  the  case  of  output 
modes  the  vocal  mode  should  be  identified.   This  follows 
from  the  fact  that  if  no  obvious  manual  action  occurs  then 
the  information  processing  channels  activated  must  be  the 
same  as  those  used  for  a  vocal  response. 

(4)  Activity  Chart ;   This  chart  is  the  graphic 
form  of  time-based  activity  log.   See  Figure  3  below 
[Mountford,  North,  Metz,  and  Graf f under , 1982 :p.  133.   These 
charts  are  particularly  important  to  the  subsequent 
screening  of  proposed  tasks  for  voice  implementation  since 
they  provide  quick  information  on  which  tasks  are  performed 
in  conjunction  with  the  flight  task  thereby  creating  a  dual 
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tasking  situation.   This  information  will  be  necessary 
during  the  optimization  portion  of  this  model. 
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Figure  3.   Activity  Chart 
(Adapted  From  Mountford,  North,  Metz,  and  Graffunder  (1982) 


b.   Development  of  Speech  I/O  Candidate  Task  List: 
After  producing  the  task  analysis  for  all 
pertinent  scenarios,   an  initial  list  of  potential 
candidates  for  conversion  to  voice  should  be  compiled. 
Each  task  should  be  evaluated  by  the  following  series  of 
filters.   These  filters  provide  the  first  gross  look  at  a 
given  task  to  judge  its  suitability  for  voice. 
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(1)  verbal  Task  Filter:   This  first  filter  asks 
whether  or  not  the  task  can  be  classified  as  one  using  the 
verbal  information  processing  channels  (see  Footnote  1).   If 
the  answer  is  yes,  it  will  continue  through  the  filters 
which  follow.   If  no,  it  will  be  rejected  immediately . 
Currently,  only  verbal  tasks  benefit  from  conversion  to 
voice  synthesis/recognition. 

(2)  Speech  Recognition  Filter;   This  filter  is 
used  to  decide  whether  a  task  would  be  a  good  candidate  for 
speech  recognition.   The  questions  for  this  filter  were 
developed  in  the  Honeywell  B-52  study,  further  refined  in 
the  Honeywell  F-18  study  and  are  provided  in  flowchart  form 
as  Figure  4  [Mountford,  North,  Metz,  and  Graf f under , 1982 : p . 
20]  . 

One  additional  question  has  been  added  to 
this  filter.   The  first  question  asks  if  the  task  is 
currently  accomplished  with  a  by  pilot  speech.   If  the 
answer  is  yes,  it  will  be  immediately  rejected  since  it 
already  uses  voice  output.   If  no,  it  will  continue  through 
this  filter. 

The  question  concerning  the  number  of  times 
the  task  is  performed  during  the  mission  has  been 
transferred  to  its  own  filter  so  that  this  it  can  be  more 
efficiently  handled. 


19 


ACCEPT 


VES 


REJECT  TASK 


NO 


^    REJECT  TASK 


JiO_ 


ACCEPT 


Ji£L 


ACCEPT 


REJECT 
TASK 


Figure  4.   Speech  Recognition  Filter 
(Adapted  From  Mountford,  North,  Metz,  and 
Graffunder  (1982)  and  Modified  By  Author) 


(3)  Speech  Generation (Synthesis )  Filter:   This 
filter  is  used  to  decide  whether  a  task  would  be  a  good 
candidate  for  speech  synthesis.   A  flowchart  for  the 
following  questions  is  provided  in  the  Honeywell  B-52  study 
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[North  and  Lea,1982:p.  21].   It  should  be  noted  that 
question  one  was  added  to  the  filter  by  this  author. 

1.  Is  the  task  currently  handled  by  speech  input  to  the 
pilot?   If  yes,  reject  task.   If  no,  continue  to 
evaluate . 

2.  Can  information  be  transmitted  to  pilot/co-pilot  as  a 
short  phrase?   If  no,  reject  task.   If  yes,  continue 
to  evaluate. 

3.  Does  it  occur  during  visual  time-sharing?   If  yes, 
accept  task.   If  no,  continue  to  evaluate. 

4.  Can  it  be  used  as  an  information  display  that 
currently  does  not  occur  in  the  central  anthropometric 
visual  zone?   If  yes,  accept  task.   If  no,  continue  to 
evaluate . 

5.  Can  it  be  used  together  with  speech  recognition  in  an 
interactive  dialogue  format?  If  yes,  accept.  If  no, 
reject . 

(4)  Technical  Feasibility:   This  filter  is  used 

to  decide  whether  a  task  can  be  implemented  as  part  of  the 

voice  system  from  a  technical  standpoint  (e.g.,  if  a  task  is 

using  equipment  not  controlled  through  the  multi-plexer 

(mux),  the  voice  system  cannot  access  it  and  the  task  will 

be  dropped  from  the  candidate  task  list).   This  is  a 

question  which  must  have  a  yes  or  no  answer  from  an 

engineering  standpoint.   Therefore,  each  task  which  was 

accepted  by  either  the  Speech  Recognition  Filter  and/or  the 

Speech  Generation  Filter  should  be  evaluated  technically. 

This  procedure  differs  from  the  Honeywell 

studies  which  address  this  issue  from  the  standpoint  of  a 

feasibility  scale  which  is  judged  to  provide  little 

additional  information  and  only  serves  to  obscure  the 

analysis  by  adding  an  additional  factor.   In  fact,  the  use 
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of  such  a  scale  could  lead  to  accepting  a  task  which  cannot 
be  implemented . 

(5)  Repetition  Filter:   At  this  point,  the  tasks 
which  have  successfully  passed  through  the  previous  filters 
should  be  grouped  according  to  equipment  used,  input/output 
mode,  number  of  steps  to  complete,  and  anthropometry  index. 

This  is  done  so  that  the  frequency  with 
which  the  same  actions  are  taken  can  be  judged\   Once  this 
grouping  has  been  compiled,  the  decision  maker  must  decide 
how  many  times  a  task  must  be  repeated  before  it  becomes 
advantageous  to  consider  conversion  to  voice  modes. 

Once  this  decision  has  been  made,  only  those 
tasks  which  occur  with  the  proper  frequency  will  be  placed 
on  the  Candidate  Task  List. 

(6)  Candidate  Task  List:  Once  a  task  has  been 
accepted  as  a  candidate  for  voice  synthesis  and/or 
recognition  it  should  be  placed  on  the  Candidate  Task  List 
which  is  the  output  for  this  module.  Along  with  each  task 
the  current  input/output  modes  and  the  anthropometry  index 
should  be  listed.  An  example  of  the  procedure  described  in 
this  module  is  included  in  the  Appendix. 


3This  handling  of  task  repetition  differs  from  the  Honeywell 
studies  in  that  instead  of  being  part  of  the  Speech  Recognition 
Filter,  it  is  handled  as  a  separate  filter.  The  reason  for  this 
change  is  that  grouping  the  tasks  as  described  above  prior  to 
asking  this  question  simplifies  the  process  considerably  when 
this  procedure  is  put  into  practice. 
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2.   Module  Two—Pilot  Input 

Pilot  involvement  in  the  task  selection  process  is 
considered  essential  by  NADC  [Interview  With  Warner,  1987]. 
Pilots  possess  a  wealth  of  knowledge  and  experience,  which 
if  properly  used,  will  greatly  enhance  this  selection 
process.   In  addition,  the  involvement  of  the  pilots  should 
result  in  greater  acceptance  of  this  new  technology. 

Both  of  the  Honeywell  Studies  [North  and  Lea, 1982 
and  Mountford,  North,  Metz,  and  Graffunder,  1982]  address 
the  assignment  of  a  pilot  utility  rating  to  those  tasks 
which  are  contained  in  the  Candidate  Task  List  produced  in 
module  one  above. 

In  the  Honeywell  F/A-18  study  [Mountford,  North, 
Metz,  and  Graffunder,  1982],  it  was  suggested  that  pilots  be 
given  a  questionnaire  which  contains  questions  requiring 
checks  in  blocks  which  are  scaled  on  an  ordinal  basis  from 
one  (never  useful)  to  six  (always  very  useful).   The 
following  example  is  taken  directly  from  that  study. 

Example:   "Descent  Checklist" 

12      3      4      5      6 
Never  Always  very 

Useful  Useful 

Sometimes 
Useful 

Responses  should  be  compiled  to  show  both  how  many 

pilots  selected  a  given  rating  and  the  average  rating.   This 
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allows  the  decision  maker  to  choose  either  the  average  or 
that  rating  that  was  picked  most  often  (the  mode)  as  the 
combined  utility  rating  to  be  used  in  the  optimization 
module.   The  following  example  illustrates  this  process. 

Example:   "Descent  Checklist" 

12     3     4     5     6 
Never  Always  very   Average 

Useful 1_   4     8     15*    12   Useful        4.83 

Sometimes 
Useful 
*Indicates  mode 

In  this  example,  there  is  very  little  difference 
between  the  average  and  the  mode  even  though  these  two 
figures  could  vary  greatly. 

In  evaluating  the  feedback  provided  by  this 
questionnaire,  strong  consideration  should  be  given  to 
deleting  any  task  from  the  candidate  list  which  receives 
ratings  which  fall  primarily  in  the  1  and  2  range.   If  not 
completely  eliminated,  these  tasks  could  still  be  selected 
in  the  optimization  module  if  other  weighting  factors  are 
high  enough.   In  most  cases  it  would  be  counter  productive 
to  provide  a  voice  task  which  the  pilots  have  already 
indicated  would  not  be  useful. 

The  output  of  this  module  is  a  list  of  candidate 
tasks  along  with  the  assigned  pilot  utility  rating  for  each. 
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3 .   Module  Three—Performance  Specifications 

The  goal  of  this  portion  of  the  model  is  to 
establish  performance  specifications  which  can  be  used  in 
Module  Four  to  maximize  workload  reduction  (reaction  time 
savings ) . 

These  performance  specifications  must  include  both 
the  single  and  dual  task  conditions.   The  reason  for 
inclusion  of  the  single  task  condition  is  that  the  mission's 
activity  chart  produced  in  module  one  includes  time  frames 
in  which  the  pilot/co-pilot  is  not  engaged  with  the  flight 
task,  thus  becoming  a  single  task  situation. 

In  establishing  performance  specifications,  it  is 
first  necessary  to  determine  whether  the  numbers  should 
reflect  the  worst  case,  best  case  or  expected  (average) 
case.   This  must  be  decided  on  an  individual  case  basis. 

One  method  for  establishing  these  specifications  is 
demonstrated  using  the  data  from  the  Sandry  study 
[ Sandry, 1982 ] .   As  previously  mentioned,  reaction  times  and 
root  mean  squared  error  rates  were  gathered  through  use  of 
the  Navy's  F-18  simulator.   The  pilots  used  were  from  the 
local  flying  club  and  had  an  average  of  nine  years 
experience  . 


Since  F-18  pilots  are  used  for  data  gathering  efforts  only 
on  advanced  high  level  projects,  the  experience  level  of  the 
pilots  used  by  Sandry  was  appropriate. 
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For  the  purposes  of  this  model,  workload  is  defined 
by  reaction  time.   This  means  that  the  reduction  of  the 
amount  of  time  necessary  to  complete  a  task  directly 
translates  into  a  reduction  of  workload  (reaction  time 
savings)  for  the  pilot/co-pilot. 

Prior  to  using  the  data  collected  by  Sandry,  its 
validity  for  the  purpose  of  establishing  performance 
specifications  was  evaluated.   A  linear  regression  model  was 
used  for  this  analysis.   Using  the  Statgraphics  Package  by 
STSC,  regressions  were  run  for  single  verbal  task  reaction 
times,  single  spatial  task  reaction  times,  dual  verbal  task 
reaction  times,  dual  spatial  task  reaction  times,  dual 
verbal  task  root  mean  square  values,  and  dual  spatial  task 
root  mean  square  values.   The  results  were  as  follows: 

1.  Residuals  indicated  that  the  linear  regression  model 
was  appropriate  for  the  reaction  times  but  not  the 
best  choice  for  root  mean  square  error  rates  (Figure 
5).   However,  since  it  is  only  necessary  to  judge 
general  trends  for  the  error  rate  (which  were 
adequately  identified  by  the  linear  regression),  no 
additional  models  were  evaluated. 

2.  Verbal  tasks  but  not  spatial  tasks  were  benefited  by 
conversion  to  voice  input  and/or  output  modes. 
Therefore,  no  performance  specifications  will  be 
developed  for  spatial  tasks. 

3.  Error  rate  declines  when  voice  input  and/or  output 
modes  are  used  (Figure  6). 

4.  In  a  single  task  condition,  a  variance  is  found  among 
the  subjects,  which  although  is  statistically 
significant,  disappears  in  the  dual  task  condition. 

5.  The  independence  of  the  variables  was  confirmed. 

In  establishing  the  performance  specifications  for 
this  demonstration,  mean  reaction  times  were  used. 
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Figure  5.   Normal  Probability  Plots  [in  sees] 
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Figure  7.   Plot  of  Reaction  Time  Variances' 


The  letters  S  and  D  were  used  to  distinguish  between  single 
and  dual  tasks,  respectively.  The  numbers  shown  with  these 
letters  identify  the  input/output  modes  as  follows: 

1  =  Visual/Manual 

2  =  Visual/Vocal 

3  =  Audio/Manual 

4  =  Audio/Vocal 
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Figure  7  shows  the  spread  of  the  reaction  times  by 
input/output  mode  and  by  single/dual  task  condition.   The 
trend  present  in  the  intervals  shown  is  consistent  with  that 
shown  by  the  means.   Therefore,  the  means  can  be  used  for 
ease  of  computation  purposes  without  distorting  the  outcome. 
The  increase  in  the  variance  between  the  single  and  dual 
task  conditions  is  explained  by  the  resource  capacity  theory 
discussed  in  Chapter  I.   The  variance  present,  while  a 
consideration,  does  not  prohibit  the  use  of  the  average  in 
establishing  performance  standards.   Table  3  below  provides 
the  performance  specifications  expected  for  tasks  in  each  of 
the  four  combinations  of  input/output  modes. 

Table  3.   AVERAGE  REACTION  TIMES 


Input/Output  Mode 
Visual/Manual  (V/M) 
Visual/Vocal   (V/V) 
Audio/Manual   (A/M) 
Audio/Vocal    (A/V) 


Single  Task 

Dual  Task 

Reaction  Times 

Reaction  Times 

10.03   [sees] 

17.91  [sees] 

8.37 

15.69 

8.57 

15.24 

6.8 

11.59 
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Tables  4  and  5  provide  the  expected  difference  in 
reaction  time  which  will  occur  if  the  input/output  modality 
is  changed. 

Table  4.   SINGLE  TASK  REACTION  TIME  SAVINGS   (seconds) 

To 

V/M      V/V     A/M  A/V 

V/M     0      1.66    1.46  3.23 

From     V/V   -1.66     0     -0.20  1.57 

A/M   -1.46    0.20     0  1.77 

A/V   -3.23   -1.57   -1.77  0 

Table  5.   DUAL  TASK  REACTION  TIME  SAVINGS  (seconds) 

To 

V/M      V/V     A/M  A/V 

V/M     0      2.22    3.67  6.32 

V/V   -2.22     0      0.45  4.10 

A/M   -3.67   -0.45     0  6.32 

A/V   -6.32   -4.10   -6.32  0 

The  potential  for  reducing  reaction  time  is  found  by 
locating  the  row  which  lists  the  input/output  mode 
combination  currently  in  use  and  combining  this  with  the 
column  indicating  the  candidate  input/output  mode. 

In  many  cases  it  will  not  be  possible  to  pull  the 
numbers  from  these  tables  and  immediately  use  them.   It  is 
first  necessary  to  relate  the  task  at  hand  to  those 
performed  in  the  Sandry  study.   The  most  straight  forward 
way  to  accomplish  this  is  to  consider  the  number  of  steps 
necessary  to  perform  the  task  in  question.   The  Sandry  tasks 
required  four  steps  for  completion.   Therefore,  the  reaction 
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time  for  the  task  under  consideration  can  be  obtained  by 
multiplying  the  tabled  reaction  time  by  the  ratio  calculated 
by  dividing  the  number  of  steps  necessary  to  complete  the 
current  task  by  four. 

Tables  3,  4,  and  5  represent  the  type  of  output 
expected  from  this  module. 

4 .   Module  4--Optimization 

This  portion  of  the  model  represents  the  heart  of 
the  total  systems  approach  to  implementation  of  voice 
synthesis/recognition  in  the  cockpit.   The  goal  of  this 
module  is  to  select  that  combination  of  candidate  tasks  with 
associated  input/output  modes  which  will  provide  the  largest 
reduction  in  workload  without  overloading  the  voice  channels 
used  by  the  voice  input/output  modes. 

Linear  programming  is  proposed  as  the  preferred 

method  for  accomplishing  this  objective.   This  method  was 

developed  as  a  means  of  dealing  with  situations  containing  a 

large  number  of  variables  with  associated  constraints.   As 

stated  by  the  Operations  Analysis  Group  (1984), 

Problems  of  allocation  arise  whenever  there  are  a  number 
of  activities  to  be  performed  but  limitations  on  either 
the  amount  of  resources  or  the  manner  in  which  they  may  be 
allocated  prevents  accomplishment  of  each  separate 
activity  in  the  most  effective  way  conceivable .. .a 
powerful  technique  that  has  been  developed  to  solve  such 
problems  is  called  mathematical  programming.   When  the 
problem  can  be  formulated  within  a  mathematical  framework 
in  such  a  way  that  it  becomes  one  of  maximizing  or 
minimizing  a  linear  expression  subject  to  certain  linear 
constraints,  the  technique  is  known  as  linear  programming. 
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Likewise,  this  is  an  allocation  problem  and  it   can 
be  formulated  within  the  "mathematical  framework"  of  a 
linear  function  with  linear  constraints.   Such  a 
mathematical  formulation  follows. 

MaximizeEj  Ej  Ek  Ej  T^kl  X^^  (2-1) 

such  that 

Li    ^k1!  Rijkl  <=  CJ  <2-2) 

£k  Xijkl  <=  1  (2-3) 

*ijkl   =   Xijk2     =    •  •  •    =   xijki  (2-4) 

where 

i  =  Candidate  Task 
j  =  Mission 
k  =  Input/Output  Mode 

1  =  Type  of  Task  (single  or  dual  condition) 
Tijkl   =  Utility  Value   Of  Task  i,  Mission  j, 

Input/Output  Mode  k,  and  Type  of  Task  1 

xiikl  =  Tne  Varia°le  <?f  Choice 

R-^j   =  Reaction  Time   For  Task  i,  Mission  j, 

Input/Output  Mode  k,  and  Type  of  Task  1 
Cj  =  Total  Time  (in  seconds)  of  Additional 
Time  Allowed  For  Voice  Modes 
n  =  Number  of  Missions  Under  Consideration 

The  objective  equation  in  (2-1)  maximizes  the  total  reaction 

time  savings  for  all  of  the  missions  under  consideration. 

The  constraint  equation  (2-2)  restricts  the  amount  of  time 

which  can  be  added  to  the  voice  information  processing 

channels  during  a  given  mission  so  that  the  pilot  will  not 


The  utility  value  is  the  weighted  combination  of  reaction 
time  savings,  pilot  utility  and  anthropometry  index.  The 
importance  of  each  element  is  dependent  upon  the  judgement  of  the 
decision  maker.  One  possible  weighting  scheme  is  demonstrated  in 
the  Appendix. 

7 
This  figure  is  obtained  from  Tables  4  and  5  as  appropriate. 
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become  overloaded  in  that  area.   This  value  is  at  the 
discretion  of  the  decision  maker.   Constraint  equation  (2-3) 
limits  the  selection  of  the  input/output  mode  to  only  one 
for  each  task.   Lastly,  constraint  equation  (2-4)  insures 
that  the  same  combination  of  tasks  will  be  selected  for  each 
mission.   This  constraint  is  deleted  when  the  missions  do 
not  contain  identical  tasks.   When  this  occurs  it  will  be 
necessary  to  evaluate  the  missions  separately  to  establish 
the  proper  mix  of  tasks.   However,  in  general,  the  same 
tasks  will  be  present  in  every  mission. 

Any  good  linear  programming  package  will  be  able  to 
solve  the  above  maximization  problem,  thereby  supplying  the 
system  designer  with  the  best  initial  combination  of  tasks. 
This  combination  of  tasks  will  represent  the  largest 
workload  savings  possible  while  not  overloading  the  pilots 
voice  interactive  channels.   The  LINDO  (Linear,  INteractive, 
Discrete  Optimizer)  software  package  by  LINDO,  Inc  is  used 
in  the  Appendix  to  solve  an  example  problem. 

C.   MODEL  SUMMARY 

This  model  has  intentionally  been  designed  to  be  general 
in  nature  so  that  it  is  easily  applied  to  any  aircraft 
cockpit.   The  procedures  described  are  specific  enough  to 
guide  the  system  designer  while  retaining  the  flexibility 
necessary  to  address  the  needs  of  the  individual  aircraft. 
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In  summary,  this  model  consists  of  a  series  of  questions 
which  evaluate  the  potential  of  all  tasks  performed  during 
all  of  the  aircraft's  missions.   Those  tasks  which  are 
considered  as  good  candidates  for  conversion  to  voice 
synthesis/recognition  are  then  rated  by  the  pilots  for 
usefulness  resulting  in  a  pilot  utility  index  for  each.   The 
tasks  are  then  evaluated  by  an  optimization  program  using 
the  performance  specifications  contained  in  Module  Three  to 
determine  what  combination  of  tasks  will  provide  the 
greatest  reduction  in  pilot  workload  without  overloading  the 
voice  channels. 
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Ill  .  FUTURE  STUDY 

A.   FURTHER  DEVELOPMENT  OF  MODEL'S  INDIVIDUAL  MODULES 

As  stated  in  the  overview  of  the  model,  individual 
modules  representing  distinct  portions  of  the  process  have 
been  used  in  order  to  facilitate  the  refinement  process  for 
this  model. 

Module  One  is  fairly  well  complete  but  the  processing  of 
this  portion  would  be  greatly  enhanced  through  use  of 
automation.   A  computer  program  (possibly  in  Fortran)  could 
be  designed  which  would  ask  the  appropriate  filter  questions 
and  then  generate  the  Candidate  Task  List  as  the  final 
output . 

Module  Two  requires  pilot  input  and  could  also  be 
greatly  enhanced  through  the  use  of  a  computer  program.   The 
survey  form  could  be  automatically  generated  from  the 
Candidate  Task  List  prepared  as  part  of  module  one. 

Module  Three  could  be  improved  in  several  ways. 
Although  useful,  better  data  on  reaction  times  could  be 
collected  for  use  in  establishing  performance  specifications 
by  using  actual  F-18  pilots.   Even  if  F-18  pilots  could  not 
be  obtained,  additional  tasks  could  be  tested  on  the 
simulator  in  hopes  of  obtaining  more  accurate  reaction 
times.   In  the  mean  time,  possibilities  for  a  more  reliable 
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method  of  converting  the  values  In  Tables  3,  4,  and  5  could 
be  explored. 

Module  Four  will  work  well  with  any  good  optimization 
package  but  more  study  into  different  weighting  schemes 
could  prove  to  be  valuable.   Both  modules  three  and  four 
would  benefit  from  being  automated  in  a  complete  package 
with  Modules  One  and  Two. 

B.   TRAINING  IMPLICATIONS 

The  training  issue  should  be  addressed  for  two  reasons. 
Firstly,  the  Sandry  study  (1982)  showed  a  significant 
reduction  in  reaction  time  for  verbal  tasks  which  were 
converted  to  voice  synthesis/recognition  input/output  modes 
after  only  a  small  amount  of  training  and  practice.   More 
dramatic  results  could  occur  with  additional  practice  by  the 
pilots.   While  pilots  are  practiced  to  the  point  of  an 
automated  response  for  visual  and  manual  actions,  this  is 
not  true  for  audio  and  vocal  ones.   Further,  testing  in  this 
area  could  produce  very  favorable  results  which  would  lead 
to  a  greater  recognition  of  voice  interaction  in  the  cockpit 
as  a  valuable  asset  for  pilots. 

Secondly,  methods  for  instructing  pilots  in  the  use  of  a 
voice  interactive  system  should  be  explored  so  as  to 
facilitate  the  training  process. 
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C.  SYNTAX  DEVELOPMENT 

Selection  of  the  words  (syntax)  to  be  used  with  the 
voice  interactive  system  must  be  made  carefully.   These 
words  must  be  meaningful  to  the  pilot  as  well  as 
distinguishable  from  each  other  by  the  recognition  system. 
Words  which  sound  too  much  alike  would  lead  to  a  higher 
recognition  error  rate. 

Pilot  input  into  the  selection  process  would  not  only 
result  in  a  superior  vocabulary  but  would  lead  to  greater 
pilot  acceptance. 

D.  SPATIAL  TASKS 

At  this  time  only  verbal  tasks  have  been  considered  as 
candidates  for  voice  synthesis/recognition.   Literature  in 
the  field  as  well  as  the  Sandry  study  (1982)  have  clearly 
demonstrated  the  fact  that  tasks  processed  by  those 
information  processing  channels  which  are  utilized  by 
spacial  tasks  do  not  benefit  by  conversion  to  voice 
input/output  modes. 

However,  the  possibility  does  exist  that  some  spatial 
tasks  could  be  converted  so  that  the  verbal  information 
processing  channels  would  be  used  to  perform  the  task.   If 
this  could  be  accomplished  then  conversion  to  voice  modes 
could  prove  beneficial. 
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For  example,  a  target  tracking  task  could  be  designed 
such  that  the  pilot  could  move  a  cursor  to  a  designated  grid 
by  stating  a  letter-number  combination  rather  than  using  a 
joystick  to  move  the  cursor  to  the  same  area. 

Creative  designing  of  cockpit  tasks  could  greatly 
enhance  the  use  of  the  voice  interactive  system. 
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APPENDIX 

EXAMPLE  OF  MODEL  PROCEDURE 

The  purpose  of  this  example  is  to  demonstrate  step  by 
step  the  procedures  described  in  the  model  contained  in 
Chapter  II.   The  treatment  of  the  included  tasks  has 
deviated  from  fact  in  some  instances  so  that  various  aspects 
of  the  model  could  be  more  fully  illustrated.   The 
information  for  this  example  has  been  obtained  from  the  Task 
Narratives  contained  in  Appendix  A  of  the  Honeywell  B-52 
Bomber  Study  [North  and  Lea,  1982].   In  this  example  the 
pilot's  job  will  be  analyzed. 

A.   MODULE  ONE 

Since  the  Mission  Scenario  and  the  Task  Narrative  are 
self-explanatory,  this  example  will  begin  with  the  Time- 
based  Activity  Logs  taken  directly  from  the  Honeywell  study 
and  contained  here  as  Tables  6,  7,  8,  and  9.   As  indicated 
by  the  logs,  the  mission  segments  are  Takeoff,  Lowlevel,  Air 
Refueling,  and  Recovery.   These  segments  are  organized  into 
two  missions.   Mission  one  consists  of  Takeoff,  Air 
Refueling,  and  Recovery  while  mission  two  consists  of 
Takeoff,  Lowlevel,  and  Recovery. 
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Activity  charts  have  not  been  prepared  for  this  example 
but  it  should  be  noted  that  single  and  dual  tasking  has  been 
considered  during  this  procedure  as  appropriate. 

The  next  step  in  the  process  is  to  screen  each  task 
listed  on  the  Time-based  Activity  Logs  for  inclusion  on  the 
Candidate  Task  List.   It  was  decided  that  a  spreadsheet 
approach  would  best  suit  the  procedure  so  Lotus  1-2-3  was 
chosen  for  this  purpose.   Tables  10  and  11  contain  the 
spreadsheets  prepared  in  the  process  of  screening  tasks 
included  in  missions  one  and  two  respectively.   A  step  by 
step  explanation  of  the  process  follows. 

The  first  step  was  to  determine  if  the  task  was 

0 

verbal  .   If  the  answer  was  affirmative,  "YES"  was  placed 
in  the  appropriate  column.   If  the  answer  was  negative,  the 
task  was  dropped  from  further  consideration  and  "Reject"  was 
entered.   Once  a  task  was  rejected  three  dashes  were  placed 
in  the  remaining  columns  to  indicate  that  this  task  was  no 
longer  under  consideration  for  voice  implementation. 
Next,  each  task  was  processed  through  the  Speech 
Recognition  Filter.   In  response  to  the  question  of  whether 
the  output  mode  was  currently  vocal,  either  "NO"  or  "Reject" 
was  entered.   If  a  "NO"  response  was  entered  the  question  of 
whether  the  task  was  manual  discrete  was  asked.   The  entries 


» 

This  term  is  defined  in  footnote  1. 
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were  either  "YES"  or  "Reject".   If  the  answer  was  yes  then 
the  question  was  whether  the  task  occurred  during  time- 
sharing.  The  entries  were  then  either  "Accept"  or  "NO".   If 
no,  then  the  task  continued  through  the  filter.   Next,  the 
question  was  asked  about  whether  the  Anthropometry  Index  was 
greater  than  zero.   The  entries  were  either  "Accept"  or 
"NO".   For  "NO"  answers,  the  last  question  about  whether  the 
task  resulted  in  information  retrieval  was  asked.   The 
entries  were  either  "Accept"  or  "Reject". 

In  this  example,  no  questions  were  necessary  after  the 
time-sharing  question  although  all  of  the  columns  were 
included  for  completeness.   Lastly,  all  of  the  accepted 
tasks  were  indicated  as  such  by  an  "X"  placed  in  the 
appropriate  column  following  the  Speech  Recognition  Filter. 

Next,  the  Speech  Generation  Filter  was  applied.   The 
questions  in  this  filter  were  processed  much  the  same  as  for 
the  Speech  Recognition  Filter  and  should  thus  be  self- 
explanatory. 

The  last  question  included  on  the  spreadsheet  dealt  with 
the  technical  feasibility  of  the  tasks  accepted  by  the 
previous  filters.   If  the  task  was  accepted  by  either 
filter,  it  was  screened  for  technical  feasibility.   If  the 
task  was  not  rejected  it  was  listed  on  the  Preliminary 
Candidate  Task  List  shown  in  Table  12. 
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Table    12.       PRELIMINARY    TASK    LIST 


MISSION  OK— flIP  REFUELING 

Task   Brnf 

Takeoff 
REVIEW  TAKEOFF  CHECKLIST 
REIERSES   BOWES 
SET  ENGINE   STALL   PREVENTION   SWITCH   TO 

CLIMATIC 
CALLS   "70   KNOTS.    NOW 
CALLS   *COMMITTE0" 

SELECTS  FLIGHT   OIPECTOB  FOB  HEADING 
■HATER-OUT   EPR" 

CALLS  AFTEB   TAKE0FF-CLIM8  CHECKLIST 
SELECTS  FLIGHT  OIPECTOB  FOB  NAV 

GUIDANCE 
ENGAGES  AUTOPILOT   TO  CAPTURE  OESIBEO 

HEROINE 
BESETS  ALTINETEBS 
SET   THRUST  GATE 

Air  Refueling 
"LEVEL  AT  FL290" 
"BEGIN  VTJUB   TURN" 
"VISUAL  CONTACT" 
•GET  PBEP  CNTCT  CL" 
"LOUD  AND   CLEAR" 
DISENGAGE   AP 
SET  EVS  FLIP  VIDEO 
"SPEED  270,    CLIMBING* 
AP    TO  AR  MODE 
HATCH  AR  LITES 
•FINAL   CONNECT* 
"BRKAUAY   PRAC" 
"BREAKAWAYS" 
DUMP   HP   AP   MOOE 
"CLEAR   TANKER" 
"GET   POST   REF   CL" 


MISSION  TWO— LQMLEVEL 


Recovery 
CALL  DESCENT  CLIST 
SET  ALT 
SET  AIRBRAKES 
DUMP  AP 

DRAG  CHUTE 
CHECK  ALL  HYDRAULIC 
CHECK  CROSSUINO  CRAB 
CALL  CHECKLIST 


Task" 

Teska 

Nuaber 

Task  Bri.f 

TaJeeofF 

Nuaber 

1 

REVIEWS  TAKEOFF  CHECKLIST 

1 

2 

RELEASES  BRAKES 

2 

3 

SET  ENGINE  STALL  PREVENTION  SWITCH  TO 
CLIMATIC 

3 

4 

CALLS  "70  KNOTS.  NOW" 

4 

4 

CALLS  "COMMITTED" 

4 

S 

SELECTS  FLIGHT  DIRECTOR  FOP  HEADING 

5 

4 

•HATER-OUT  EPR" 

4 

6 

CALLS  AFTER  TAKEOFF-CLIMB  CHECKLIST 

6 

S 

SELECTS  FLIGHT  OIRECTOR  FOR  NAV 
GUIDANCE 

S 

7 

ENGAGES  AUTOPILOT  TO  CAPTURE  OESIREO 
HEADING 

7 

a 

BESETS  ALTIMETERS 

a 

9 

SET  THRUST  GATE 

Lowleval 

9 

4 

•GET  LL  OESC  CL" 

4 

4 

TURN  ON  EVS  TA  VIO 

8 

4 

BAD  ALT  TO  BOO  FT 

8 

4 

SET  ALTS 

8 

4 

DUMP  AP 

13 

10 

AP  TO  LL 

10 

a 

CHK  ALTS 

IS 

4 

SET  ALTS 

8 

10 

CHECK  SEAR,  FLAPS  UP 

IS 

11 

•GET  TA  CHCKLST  UP' 

4 

4 

.  SET  ALT  HOLO  AP 

16 

4 

CHK  ALTS 

IS 

4 

•YOUR  RIPPLANE" 

4 

13 

COMPARE  STAB  MODES 

15 

4 

SET  STAB  REF  SEL. 

S 

4 

PROFILE  SET 

8 

REAO  RAO  ALT. 

15 

SET  CL  PLANE  CAL  VAL 

a 

SET  ALTS 

8 

SET  STAB  REF  SEL 

9 

SET  CL  PLANE  TO  ALT 

5 

SET  RAOALT  CURSOR 

a 

•I  HAVE  THE  AC" 

4 

TR5RNS  BOMB  PEL  LITES 

17 

Recovery 

6 

CALL  DESCENT  CLIST 

6 

12 

SET  ALT 

12 

12 

SET  AIRBRAKES 

12 

13 

OUMP  AP 

13 

14 

DRAG  CHUTE 

14 

IS 

CHECK  ALL  HYDRAULIC 

15 

15 

CHECK  CROSSWINO  CRAB 

IS 

6 

CALL  CHECKLIST 

6 

■  TASK  NUMBER  KEY 

Reviews  Takeoff  Checklist 

Releases  Brakes 

Set/Select/Engage/Oisengege  (A/M  2) 

Instruction 

Set/Select/Engege/Oisengage  (V/M  3) 

Checklist 

Set/Select/£ngege/Oisengege  (V/V  3) 

Set  ^Select. 'Engage/ Oisenqene  (V/M  1) 

Set/Select/Engage/Oisengege  (V/M  2) 

BP/AR  (V/M  3) 

BP/AR  (V/V  2) 

Sat/Select /Engage/Engage    (A/M    1) 

AP/AR   (V/M   1> 

Oreg   Chute 

Checks 

Set/Select/Engege/Oisengage    (V/M  3) 

Set/Select. tngaqe/Oisengage    (V/V  2) 


1 

2 

3 

4 

S 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 
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At  this  point  the  tasks  were  then  grouped  into  types  of 
tasks  based  on  whether  they  accessed  the  same  equipment,  had 
the  same  input/output  mode,  required  the  same  number  of 
steps  to  complete  and  were  assigned  the  same  anthropometry 
index.   Numbers  with  a  corresponding  key  were  used  to 
identify  like  tasks.   Each  task  which  occurred  more  than 
once  in  either  mission  was  included  on  the  Candidate  Task 
List  (Table  13) . 

Table  13.   CANDIDATE  TASK  LIST 

1.  Verbal  Instruction 

2.  Set/Select/Engage/Disengage  (V/M  3)  9 

3.  Checklist 

4.  Set/Select/Engage/Disengage  (V/M  1) 

5.  Set/Select/Engage/Disengage  (V/M  2) 

6.  Adjust  AP/AR  (V/M  3) 

7.  Set/Select/Engage/Disengage  (A/M) 

8.  Adjust  AP/AR  (V/M  1) 

9 .  Checks 

B.   MODULE  TWO 

This  module  requires  that  pilots  be  surveyed  and  a  Pilot 
Utility  Index  (PUI)  be  assigned  to  each  of  the  tasks  listed 
in  Table  13.   For  the  purposes  of  this  example,  it  was 
assumed  that  the  pilots  had  been  questioned  and  the  PUI(s) 
listed  in  Table  14    obtained. 


Q 

The  information  in  parentheses  indicate  the  input/output 
mode  and  anthropometry  index  of  the  task  listed.  This 
information  is  provided  when  tasks  with  identical  titles  are 
included . 

50 


Table  14.   PILOT  UTILITY  INDEX 

Task  PUI 

1.  Verbal  Instruction  4.2 

2.  Set/Select/Engage/Disengage  (V/M  3f                                 5.2 

3.  Checklist  3.5 

4.  Set/Select/Engage/Disengage  (V/M  1)              4.8 

5.  Set/Select/Engage/Disengage  (V/M  2)              4.8 

6.  Adjust    AP/AR    (V/M    3)  5.5 

7.  Set/Select/Engage/Disengage  (A/M)                4.8 

8.  Adjust  AP/AR  (V/M  1)  5.0 

9.  Checks  5.5 


C.  MODULE  THREE 

Module  Three  contains  the  performance  specifications 
developed  for  use  in  completing  the  optimization  in  module 
four.   A  conversion  factor  was  obtained  for  each  task  listed 
on  Table  13  based  on  the  number  of  steps  the  each  task 
requires . 
D.   MODULE  FOUR 

The  optimization  problem  for  this  example  has  the  same 
formulation  as  described  in  module  four  of  the  model 
description  contained  in  Chapter  II.   The  only  exception  is 
that  the  type  of  task  indicator  is  not  necessary  since  no 
single  tasks  remained  for  consideration  after  the  filters 
were  applied. 

As  mentioned  in  Chapter  II,  the  LINDO  software  package 
by  LINDO,  Inc  was  used  to  solve  this  problem.   Figure  8 


The  information   in  parentheses  indicate  the   input/output 

mode   and   anthropometry    index   of  the   task    listed.     This 

information   is  provided   when   tasks  with   identical  titles   are 
included . 
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contains  the  LINDO  problem  formulation.   For  each  mission,  C 
(the  maximum  total  additional  time  allowed  for  voice 
processing),  was  calculated  by  deciding  the  total  amount  of 
time  which  could  be  spent  in  voice  information  processing 
channels  during  a  mission  and  subtracting  from  it  the  time 
currently  consumed  in  these  channels  by  non-candidate  tasks. 
The  time  was  calculated  by  using  the  conversion  factor 
calculated  above  along  with  the  reaction  times  contained  in 
Table  3.   The  utility  value  T^was  calculated  by  summing 
the  reaction  time  savings  (conversion  factor  x  appropriate 
Table  5  entries  x  number  of  times  tasks  performed  during  the 
mission),  pilot  utility  index,  and  the  anthropometry  index. 

Figure  9  contains  the  solution  produced  by  the  LINDO 
package.   Tasks  two  through  eight  were  selected  to  be 
converted  to  the  audio/vocal  mode  while  task  nine  was 
selected  for  conversion  to  the  audio/manual  mode.   Since 
task  one  contains  a  fractional  value,  it  is  dropped  from 
conversion  consideration.   This  combination  of  tasks  are  now 
used  by  the  engineer  to  begin  design  and  testing. 
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MAX  -26.  88  Xlll  ♦  4.2  X112  ♦  10.5  X113  »  61.6  X114  ♦  8.2  X211  ♦  12.64  X212 

♦  19.53  X213  ♦  27.16  X214  -  3.16  X311  *  3.5  X312  •  4.85  X313  ♦  15.8  X314 
»  5.8  X411  »  10.24  X412  ♦  13.14  X413  ♦  18.44  X414  ♦  6.8  X511  ♦  9.02  X512 

♦  10.47  X513  •  12.8  X514  »  8.S  X611  f  12.94  X612  *  15.84  X613  •  21.14  X614 

-  1.54  X711  »  4.9  X712  •  5.8  X713  »  18.44  X714  *  6.0  X811  ♦  10.44  XB12 

♦  13.34  XB13  ♦  11.64  X814  •  2. OS  X911  ♦  6.5  X912  '    7.4  X913  »  14.7  X914 

-  11.34  X121  •  4.2  X122  ♦  7.35  X123  ♦  32.9  X124  »  8.2  X221  ♦  12.64  X222 

♦  19.53  X223  >  27.16  X224  -  3.16  X321  ♦  3.5  X322  ♦  4.85  X333  ♦  15.8  X324 

♦  5.8  X421  »  25.78  X422  »  38.83  X423  ♦  62.68  X424  •  6.8  X521  t  11.24  XS22 

♦  14.14  XS23  ♦  19.44  X524  ♦  8.5  X621  ♦  10.72  X622  ♦  12.22  X623  ♦14.82  X624 

-  1.54  X721  »  4.9  X722  ♦  5.8  X723  ♦  18.44  X724  ♦  6.0  X821  t  10.44  X822 

»  13.34  X823  »  18.44  X824  -  6.82  X921  ♦  6.5  X922  *    9.2  X923  »  31.11  X924 

ST 

Xlll  ♦  X112  ♦  X113  •  X114  <  1 

X211  ♦  X212  ♦  X213  »  X214  <  1 

X311  ♦  X312  ♦  X313  ♦  X314  <  1 

X411  •  X412  »  X413  ♦  X414  <  1 

XS11  ♦  X512  ♦  X513  ♦  X514  <  1 

X611  »  XS12  ♦  X613  »  X614  <  1 

X711  ♦  X712  ♦  X713  »  X714  <  1 

X811  *  X812  »  X813  ♦  X814  <  1 

X911  ♦  X912  ♦  X913  ♦  X914  <  I 

219.66  Xlll  »  219.66  X112  *  213.6  X113  •  162.26  X114  ♦ 

35.82  X211  *  31.31  X212  ♦  30.48  X213  ♦  23.18  X214  * 

53.73  X311  •  47.07  X312  »  45.72  X313  ♦  34.77  X314  ♦ 

35.82  X411  »  31.38  X412  ♦  30.48  X413  ♦  23.18  X414  * 

17.91  X511  ♦  15.69  X512  ♦  15.24  X513  ♦  11.59  X514  ♦ 

35.82  X611  ♦  31.38  X612  ♦  30.46  X613  ♦  23.18  X614  ♦ 

35.82  X711  ♦  31.38  X712  ♦  30.48  X713  ♦  23.18  X714  » 

35.82  X611  ♦  31.36  X812  ♦  30.46  X813  •  23.18  X814  ♦ 

35.82  X911  »  31.38  X912  »  30.46  X914  *    23.18  X914  <  312 

X121  ♦  X122  ♦  X123  »  X124  <  1 

X221  •  X222  •  X223  »  X224  <  1 

X321  ♦  X322  ♦  X323  ♦  X324  <  1 

X421  •  X422  ♦  X423  *  X424  <  1 

X521  •  X522  ♦  X523  ♦  X524  <    1 

X621  •  X622  •  X623  »  X624  <  1 

X721  ♦  X722  ♦  X723  ♦  X724  <  1 

X821  ♦  X822  »  X823  ♦  X824  <  1 

X921  *  X922  ♦  X923  ♦  X912  <  1 

125.37  X121  ♦  109.83  X122  •  106.68  X123  »  61.13  X124  ♦ 

35.82  X221  •  31.36  X222  •  30.48  X223  *  23.18  X224  t 

53.73  X321  ♦  47.07  X322  ♦  45.72  X323  ♦  34.77  X324  ♦ 

161.19  X421  »  141.21  X422  '    137.16  X423  ♦  104.31  X424  ♦ 

35.82  X521  ♦  31.38  X522  ♦  30.48  X523  ♦  23.18  X524  » 

17.91  X621  ♦  15.69  X622  •  15.24  X623  •  11.59  X624  » 

35.82  X721  »  31.38  X722  •  30.48  X723  ♦  23.18  X724  ♦ 

35.82  X821  *  31.38  X622  -    30.48  X823  •  23.18  X824  ♦ 

107.46  X921  ♦  94.14  X922  ♦  91.44  X924  ♦  69.54  X924  <  390 

X111-X121-0 

X112-X122»0 

X113-X123-0 

X114-X124-0 

X211-X221-0 

X212-X222-0 

X213-X223-0 

X214-X224.0 

X311-X321-0 

X312-X322-0 

X313-X323-0 

I314-X324-0 

X411-X421-0 

X412-X422»0 

X413-X423-0 

X414-X424.0 

XS11-X521-0 

X512-X522-0 

XS13-X523-0 

X514-X524-0 

X611-X621'0 

X612-X622»0 

XS13-X623-0 

X614-X624-0 

X711-X721-0 

X712-X722-0 

X713-X723-0 

X714-X724-0 

X811-X821-0 

X812-X622-0 

X813-X823'0 

X814-X824-0 

X911-X921-0 

X512-X922-0 

X913-X923-0 

X914-X924-0 

no 

Figure  8.   Lindo  Mathematical  Formulation 


Xj^     with     i  =  Candidate  Task 

j  =  Mission  Number 
k  =  Input/Output  Mode 
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LP  OPTIMUM  FOUND   AT  STEP      45 

OBJECTIVE  FUNCTION  VALUE 


1)           413.208300 

VARIABLE 

VALUE 

REDUCED  COST 

mi 

.000000 

154.809700 

X112 

.000000 

.000000 

X113 

.000000 

.000000 

XU4 

.922840 

.000000 

X211 

.000000 

.000000 

X212 

.000000 

.000000 

X213 

.000000 

.000000 

X214 

1.000000 

.000000 

X311 

.000000 

45.802280 

X312 

.000000 

31.763500 

X313 

.000000 

.000000 

X314 

1.000000 

.000000 

1411 

.000000 

76.881520 

X412 

.000000 

.000000 

X413 

.000000 

.000000 

X414 

1.000000 

.000000 

X511 

.000000 

.000000 

X512 

.000000 

.000000 

X513 

.000000 

.000000 

X514 

1.000000 

.000000 

xen 

.000000 

.000000 

X612 

.000000 

.000000 

X613 

.000000 

12.151510 

X614 

1.000000 

.000000 

X711 

.000000 

45.781520 

X712 

.000000 

31.855670 

X713 

.000000 

.000000 

X714 

1.000000 

.000000 

X811 

.000000 

.000000 

X812 

.000000 

.000000 

X813 

.000000 

.000000 

X814 

1.000000 

.000000 

X911 

.000000 

.000000 

X912 

.000000 

.000000 

X913 

1.000000 

.000000 

X914 

.000000 

.000000 

X121 

.000000 

11.340000 

X122 

.000000 

119.529700 

X123 

.000000 

106.550400 

X124 

.922840 

.000000 

X221 

.000000 

45.281520 

X222 

.000000 

33.815670 

X223 

.000000 

19.511510 

X224 

1.000000 

.000000 

X321 

.000000 

3.160000 

X322 

.000000 

.000000 

X323 

.000000 

28.277270 

X324 

1.000000 

.000000 

X421 

.000000 

.000000 

X422 

.000000 

49.875670 

X423 

.000000 

33.401510 

X424 

1.000000 

.000000 

X521 

.000000 

22.320760 

X522 

.000000 

14.367830 

X523 

.000000 

9.755755 

X524 

1.000000 

.000000 

X621 

.000000 

26.321520 

X622 

.000000 

17.075670 

X623 

.000000 

.000000 

X624 

1.000000 

.000000 

X721 

.000000 

1.540000 

X722 

.000000 

.000000 

X723 

.000000 

29.531510 

X724 

1.000000 

.000000 

X821 

.000000 

32.641520 

X822 

.000000 

21.175670 

X823 

.000000 

14.851510 

X824 

1.000000 

.000000 

X921 

.000000 

42.221520 

X922 

.000000 

21.875670 

X923 

1.000000 

.000000 

X9  2  4 

.000000 

2.041512 

Figure  9.   Lindo  Solution 
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